Skip to content

feat: wire update, publish/unpublish and job status to real backends … - #13

Open
lirantul123 wants to merge 14 commits into
masterfrom
gateway-conversion
Open

lirantul123 wants to merge 14 commits into
masterfrom
gateway-conversion

Conversation

@lirantul123

@lirantul123 lirantul123 commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

…(MAPCO-11833)

Replace the stub PATCH handlers and add a job-status endpoint, making the service a real successor to 3d-gateway + store-trigger:

  • PATCH /record/{id}: validate the update (record exists, not being-deleted, v2 partial schema, footprint, classification, unique name), reject records that exist in extractable-management (409), convert the footprint to 2D and patch the catalog.
  • PATCH /record/status/{id}: validate the record, extractable guard, then change status in the catalog.
  • GET /jobs/{jobId}: return the Jobnik job status and completion percentage.

Add CatalogCall getRecord/patchMetadata/changeStatus, an ExtractableCall client (externalServices.extractable), JobnikClient.getJobStatus, and a job module (model/controller/route mounted at /jobs). Unit and integration tests included.

Question Answer
Bug fix ✖
New feature ✔
Breaking change ✖
Deprecations ✖
Documentation ✖
Tests added ✔
Chore ✖

Rename boilerplate identity to 3d-ops-trigger across package.json, openapi,
helm chart/values, catalog-info and README. Pin Node>=24 (engines) and wire
core deps: @map-colonies/jobnik-sdk, @map-colonies/3d-shared (local file link),
zod.
Define the four 3D operations in openapi3.yaml (POST /record, DELETE
/record/{id}, PATCH /record/{id}, PATCH /record/status/{id}) and wire the
record resource (router, controller, manager) with tsyringe. Managers are
stubbed pending Jobnik/catalog integration. Replace boilerplate example
resources. Integration tests at the HTTP seam + a RecordManager unit test.
Add ValidationManager that validates ingestion metadata using zod schemas from
@map-colonies/3d-shared: core fields (new3DLayerMetadataSchema), footprint 2D/3D
(geometrySchema), and date-before-date. Invalid metadata throws AppError(400)
routed via the controller error seam. Wire the validator into RecordManager's
ingestion path. Restore AppError (now used). Unit + integration tests.
Add JobnikClient wrapping the @map-colonies/jobnik-sdk Producer.
- Ingestion: create 3D-Ingestion job + validation stage/task (+ data-extraction
  stage/task only for 3tz input).
- Delete: validate deletability (record exists, productType not QuantizedMeshDTMBest,
  productStatus UNPUBLISHED) then create a 3D-Delete job.
RecordManager ingest/delete now return real Jobnik jobs. Add externalServices.jobManager
config (schema, default, custom-environment-variables, helm). Unit + integration tests.
Add ValidationManager check that rejects an ingestion when an in-flight
Jobnik ingestion job already exists for the same productName, complementing
the catalog unique-name check (which only sees finished records).

JobnikClient.hasInFlightIngestionJob queries GET /v1/jobs by job name and
filters client-side on non-terminal status + data.metadata.productName,
paginating through all pages. Query failures raise AppError. JobnikClient
is now a singleton so RecordManager and ValidationManager share one SDK
instance instead of double-registering its metrics.

Interim until Jobnik ships native duplicate support (epic MAPCO-11833).
Add tileset validation to the ingestion flow for both 3tz and folder inputs:
read tileset.json, extract the model polygon from its bounding volume, and
require the footprint to intersect and cover the model above a configurable
threshold.

TilesetReader reads tileset.json without a zip dependency: for a 3tz it reads
the first local file entry directly via a file handle (real 3tz archives break
yauzl's central-directory parsing), guarding against data-descriptor/ZIP64
entries whose size is absent from the local header; for a folder it reads the
file from disk. Polygon extraction (region/sphere bounding volumes) lives in
@map-colonies/3d-shared. Coverage is checked with turf against
validation.percentageLimit (default 10), wired through schema, config,
custom-environment-variables and helm. Unit and integration tests included.
Return ValidationResponse {isValid, message} from the tileset polygon
extraction and footprint-intersection checks (mirroring 3d-gateway), route
every failure through a single rejectValidation() that logs the outcome and
throws AppError so the message reaches the client. Move ValidationResponse to
common/interfaces. Match 3d-gateway's coverage/far-from-model messages and add
a coverage-below-threshold test.
…(MAPCO-11833)

Replace the stub PATCH handlers and add a job-status endpoint, making the
service a real successor to 3d-gateway + store-trigger:
- PATCH /record/{id}: validate the update (record exists, not being-deleted,
  v2 partial schema, footprint, classification, unique name), reject records
  that exist in extractable-management (409), convert the footprint to 2D and
  patch the catalog.
- PATCH /record/status/{id}: validate the record, extractable guard, then
  change status in the catalog.
- GET /jobs/{jobId}: return the Jobnik job status and completion percentage.

Add CatalogCall getRecord/patchMetadata/changeStatus, an ExtractableCall client
(externalServices.extractable), JobnikClient.getJobStatus, and a job module
(model/controller/route mounted at /jobs). Unit and integration tests included.
…us and bundle (MAPCO-11833)

Bring the service closer to the 3d-gateway behaviour:
- POST /record/validate: run the full ingestion validation and return
  {isValid, message} without creating a job.
- Ingestion validation also enforces modelPath under the configured base path
  (validation.basePath) and productId uniqueness in the catalog.
- Ingestion converts the footprint to 2D before creating the job.
- Delete now sets the catalog record status to BEING_DELETED after the delete
  job is created.
- Add an openapi:bundle script (redocly) producing bundledApi.yaml.

Skipped by design (Jobnik model, not store-trigger): PV-path rewriting,
pre-minted uuid modelId, StoreTriggerCall.
…833)

- GET /record/canDelete/{id}: dry-run delete validation returning
  {isValid, message}, matching 3d-gateway's GET /models/canDelete/{recordId}.
- Rename the bundle script to bundleOpenApi and commit bundledApi.yaml
  (tracked, like 3d-gateway and store-trigger; removed from .gitignore).
…a groups (MAPCO-11833)

Restructure the routes to mirror 3d-gateway + store-trigger groupings instead of
a single /record tag:
- jobOperations: POST /jobOperations/ingestion, POST /jobOperations/delete
- jobStatus: GET /jobStatus/{jobId}
- models: POST /models/validate, GET /models/canDelete/{recordId}
- metadata: PATCH /metadata/{identifier}, PATCH /metadata/status/{identifier}

Delete is now POST with a body ({id}) like store-trigger. Each group has its own
controller and router; RecordManager and JobManager remain the shared business
models. OpenAPI, generated types, bundledApi and integration tests updated.
…-11833)

Mirror 3d-gateway's models group (renamed to our domain term "record"): all
model operations live under one records tag, metadata and jobStatus stay
separate.
- records: POST /records, POST /records/validate, GET /records/canDelete/{recordId},
  DELETE /records/{recordId}
- metadata: PATCH /metadata/{identifier}, PATCH /metadata/status/{identifier}
- jobStatus: GET /jobStatus/{jobId}

Delete reverts to a REST DELETE. Merge the jobOperations and models modules into
a single records module (recordManager moved under it); update routers,
serverBuilder, containerConfig, OpenAPI, bundledApi and tests.
…CO-11833)

Mint a modelId (uuid) at ingestion time, pass it into the Jobnik job data so
the finalizer has a stable catalog id, and tag the trace span with it. Wrap the
create flow in a try/catch that passes AppError through and maps unexpected
errors to a 500, mirroring 3d-gateway's createModel.

productSource and PV-path rewriting are intentionally omitted: productSource
does not exist in the v2 metadata schema, and path rewriting is store-trigger
mount-specific (revisit only if the Jobnik worker contract requires it).
- move record/job DTO aliases into @common/interfaces, dropping the
  duplicate UpdatePayload and the local JobStatusResponse
- consume RecordStatus/ProductType from @map-colonies/types instead of
  hardcoded status/productType literals (adds @map-colonies/types + mime-types)
- split ValidationManager into IngestionValidator, MetadataValidator and
  TilesetValidator, with shared errors.ts and a classification helper
- dedupe CatalogCall error handling behind a single request helper
Base automatically changed from tileset-extraction to master November 4, 2026 09:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants